Generalizability and usefulness of artificial intelligence for skin cancer diagnostics: An algorithm validation study

Background Artificial intelligence can be trained to outperform dermatologists in image-based skin cancer diagnostics. However, the networks' sensitivity biases and overfitting may hamper their clinical applicability. Objectives The aim of this study was explain potential consequences implementing convolutional neural networks for stand-alone melanoma diagnostics lesion triage. Methods In algorithm validation on retrospective data, we reproduced evaluated performance state-of-the-art artificial (convolutional networks) were 25,331 annotated dermoscopic images from an open-source data set (ISIC-2019) tested using a novel (AISC-2021) consisting 26,591 images. We algorithms' ability generalize new diagnostic two simulations (melanoma triage). Results algorithms performed significantly less accurate nevi, melanomas actinic keratoses AISC-2021 than ISIC-2019 (p < 0.003). Almost one-third (31.1%) misclassified during simulation, irrespective Breslow thickness. Furthermore, marked 92.7% lesions ‘suspicious’ triage which yielded specificity 99.7% 8.2%, respectively. Conclusions Although outperforms classification within setting, additional technological advances are needed before implementation.